Lessons from building a Persian written corpus: Peykare

This paper addresses some of the issues learned during the course of building a written language resource (called ‘Peykare’) for contemporary Persian. After defining five linguistic varieties and 24 different registers based on these linguistic varieties, we collected the texts for Peykare to do a linguistic analysis, including cross-register differences. For tokenization of Persian, we have proposed a descriptive generalization to normalize orthographic variations existing in texts. To annotate Peykare, we have used EAGLES guidelines, having tag hierarchies as a result. For the methodology used in annotation of Peykare, we had a semi-automatic approach. Special attention was also given to the Ezafe construction and homographs which cause difficulties in text analyses.